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Abstract 

The purpose of the study was to report the analysis results of two-year accumulative data from Research 
Academy workshops held for pre-service and in-service teachers in a southern state Hispanic-Serving Institution. 
Graduate students’ perceptions of learning through these professional development workshops were reported. 
Statistical analyses were performed. The study found consistent and significant gains of participants’ confidence 
in educational research skills and knowledge. Future directions, limitations, and implications were discussed. 

Keywords: research workshop, perception of learning, Hispanic-Serving Institution, teacher professional 
development 

1. Introduction 

Professional development for teachers has been recognized as one of the core elements to improve quality of 
teaching and learning for students (Borko & Putman, 1995; Darling-Hammond, 1992; Darling-Hammond, 1997; 
Thompson & Zeuli, 1999). Advancing knowledge and skills is considered a main purpose of education-related 
professional development (Garet, Porter, Desimone, Birman, & Yoon, 2001; Shulman & Sparks, 1992). A group 
of faculty members with different educational backgrounds in a southern state university in the United States 
with 93% of its students being Hispanic established the College of Education Research Council (COERC) in Fall 
2012 (Lu, Ward, Overton, & Shin, 2014). This Hispanic-Serving Institution (HSI) has held over two years of 
Research Academy workshops and the initial analyses from the first year’s workshops had demonstrated high 
effectiveness of workshop implementation (Lu, Ward, Overton, & Shin, 2014). The purpose of the study was to 
review and report the analysis results of 2-year accumulative data from Research Academy workshops held by 
COERC for mainly pre-service and in-service teachers in the HSI. Participating graduate students’ perceptions of 
learning through this specific type of professional development workshops, namely the research workshops, 
were investigated and reported. The paper first reviewed related literature, described the research study method, 
then reported the two-year data analysis results, and lastly discussed limitations and implications. 

2. Literature Review 

Professional development is viewed as essential to educational reform and is closely linked to teacher learning 
and improved instruction for student achievement (Desimone, 2009; Sykes, 1996). Therefore, professional 
development is interchangeably used with educational reform to some degree (Desimone, 2009; Sykes, 1996). 
Little (1993) examines in-service and pre-service teachers’ professional development as a way of educational 
reform (Little, 1993). It is important to understand the major elements of effective professional development in 
order to determine the possible effects that professional development might have on educational reform 
(Desimone, 2009). 
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2.1 Effective Professional Development 

Some research studies examine the effectiveness of professional development workshops or seminars (e.g., 
Boyle, While, & Boyle, 2004; Lu, Ward, Overton, & Shin, 2014). Several research studies suggest that 
professional development mainly emphasizes the subject matter content, curriculum, or strategy use to teach 
students in an effective way and consequently to increase students’ achievement (Cohen & Hill, 2000; Kennedy, 
1998). Penuel, Fishman, Yamaguchi, and Gallagher (2007) examined the effects of different characteristics of a 
science program on teachers’ perception about knowledge gains and ability to implement the program in the 
classroom. The results from a survey of 454 teachers highlighted the importance of coherent professional 
development experiences for successful teacher learning and program implementation. The study also 
underpinned that combined effects of time for teachers to implement the science program and technical support 
from a school or a district contributed to high-quality professional development and promoted the 
implementation of the science program. 

Content focus has been argued to be the most influential factor leading to effective professional development 
(e.g., Banilower, Heck, & Weiss, 2007; Cohen & Hill, 2000; Cohen, 1990). Many studies, based on various 
methodologies such as case studies, quasi-experiments, longitudinal studies, and experimental designs, clearly 
supported the significance of content focus (Banilower, Heck, & Weiss, 2007; Cohen & Hill, 2000; Cohen, 
1990; Desimone, Porter, Garet, Yoon, & Birman, 2002; Garet et al., 2001). In addition to content focus, 
Loucks-Horsley and Matsumoto (1999) considered immersion in inquiry as one of the essential elements for 
effective professional development, in which participants engage in an activity that they are likely to practice 
with their own students. 

Regarding effective activities that contribute to the success of professional development, previous research 
studies have found and identified the following: participant engagement in structured and collaborative 
interactions around topics, coaching or mentoring arrangements in which participants have rich opportunities to 
work with more experienced teachers, facilitated networks between teachers and groups to explore and discuss 
topics of interest, sharing information, and pursuing common goals (Darling-Hammond, 1992; Hargreaves & 
Fullan, 1992; Lieberman, 1996; Loucks-Horsley, Styles, & Hewson, 1996; Richardson, 1996). 

The effective activities or components of professional development may be associated with time involvement. 
Some researchers investigated if professional development sessions longer in duration were more effective since 
they allowed teachers to have more practice time and more time to reflect upon their teaching as part of an 
ongoing process (e.g., Shields, Marsh, & Adelman, 1998; Weiss, Montgomery, Ridgway, & Bond, 1998; Weiss, 
Banilower, & Shimkus, 2004). As a major characteristic of professional development, a handful of research 
studies examined the duration or length of the professional development and its association with the depth of 
teaching and teacher changes (Shields, Marsh, & Adelman, 1998; Weiss, Montgomery, Ridgway, & Bond, 1998; 
Weiss, Banilower, & Shimkus, 2004). Brinkerhoff (2006) examined the effects of long-duration professional 
development designed to develop teachers’ technology skills. He analyzed data from the surveys of participants’ 
self-assessment technology skills and self-efficacy, beliefs and feelings on the use of technology in classrooms, 
technology integration into the classroom, and teacher interviews. The professional development was designed to 
support teachers’ mastery of technology skills and integration practices across two academic years and this 
long-duration academy allowed teachers to overcome the issue of insufficient time to acquire a skill while 
applying it in classrooms. The results from data analysis revealed that the participants significantly gained 
technology skills and computer self-efficacy, with little change to technology integration beliefs and practice. 
The article identified two major contributing factors to the success of the professional development: (1) the 
extended nature of the academy program, and (2) teacher participants volunteered to participate in the academic 
workshop program based on their own needs or interests. 

Taken together, the core elements playing important roles in determining the effectiveness of professional 
development include the following: content focus (Cohen & Hill, 2000; Cohen, 1990; Garet et al., 2001), active 
learning (Banilower & Shimkus, 2004; Borko, 2004; Darling-Hammond, 1997), coherence, the extent to which 
knowledge gains from professional development are consistent with teachers’ knowledge and beliefs (Elmore & 
Burney, 1996), duration including both the span of time over which professional development is spread and the 
amount of time spent in the professional development (Cohen & Hill, 2000; Fullan, 1993; Guskey, 2003; 
Supovitz & Turner, 2000), and collective participation in which participation from the same school, grade, or 
department might generate potential interaction and discourse among teachers as a powerful form of teacher 
learning (Banilower & Shimkus, 2004; Borko, 2004; Desimone, 2003; Fullan, 1993). 
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2.2 The Role of Research in Teacher Education 

However, the role of research in teacher education is less explored while addressing the effectiveness of 
professional development focusing on subject matters such as science or mathematics or core elements for 
effective professional development. The main purpose of promoting research is to sharpen and maintain 
teachers’ insight and professional curiosity (Rudduck, 1985). Effective teaching involves not only the ability to 
implement new strategies, develop effective tasks, and provide assessment tools tailored to different students’ 
learning style but also the ability to apply and adopt new evidence-based research in specific classroom contexts 
(Stafford, 2006). This level of teacher quality cannot be accomplished unless teachers are knowledgeable about 
new research in their subject matter and new research methodologies and are eventually trained to be teacher 
researchers themselves for enhancing teaching practice. 

Although research is widely recognized as an essential educational component, there remains a concern about 
how to efficiently implement research in educational settings. For example, research components and skills 
should be integrated into research within the teaching contexts for effectively creating an environment in which 
research knowledge or skills are needed to complete their immediate tasks (Lovat, Davies, & Plotnikoff, 1995). 
Despite several previous studies about the significance and subsequent influential factors of professional 
development, there is a relative paucity of studies designed to implement research components in the 
professional development, and let alone to evaluate the effects of long-durational research knowledge-and-skill 
integration workshops for pre-service or in-service teachers. 

The paper augmented current literature in teacher’s research-oriented professional development by describing a 
series of professional development workshops focusing on infusing research knowledge and skills into teaching 
context. The paper further explored the participants’ perceptions of knowledge and confidence gains over the 
series of professional development and the relations between their achievement and other influential variables 
which might be related to their learning outcomes. 

2.3 Research Academy Workshops 

The Research Academy workshops were described and reported by Lu, Ward, Overton, and Shin (2014). The 
first two years of workshop series consisted of 13 workshops with various topics and details (Table 1). The 
AP AJ Academic writing workshop contained the most valid data points, with participants (N=64) while review of 
statistics has the fewest participants (N=19). The second years’ workshops’ order was changed based on the first 
year’s empirical data, in which a new framework was formed with the order of APA writing, journal reading, 
research method, IRB process, qualitative research method, and proposal writing. The highest number of the 
participants in APA writing workshop indicated the graduate students’ and novice researchers’ urgent need for 
APA format, academic writing, and the basic structure of academic paper. Therefore this workshop was offered 
first in the second year. In a similar vein, the review of statistics session was provisionally eliminated from the 
workshop series due to the issues with time conflict and the participant’s responses over the topic. 


Table 1. The order of the workshops in 2012-2014 

Y ear 1 Y ear 2 

Title Order & No. Order and No. Description 

_ (2012-2013) (2013-2014) _ 


Journal Reading 

1 (N=30) 

2 (N=49) 

APA Writing 

2 (N=64) 

1 (N=63) 

Research Methods 

3 (N=37) 

3 (N=38) 

Review of Statistics 

4 (N=19) 

- 

Qualitative Research 
Method 

5 (N=43) 

5 (N=20) 

IRB Process 

6 (N=43) 

4 (N=40) 


- Elements of journal articles 

- Abstract activity 

- Brief descriptions of APA styles 

- Definition of Plagiarism 

- References & citations 

- Introduction of research methods 

- Qualitative/quantitative/mixed-up methods 

- How to design own research 

- Introduction of basic statistics 

- how to interpret the statistical information 

- Instruction of various types of qualitative research 
methods & analyzing techniques 

- Understanding IRB process & procedures 

- The importance of research ethics 
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Proposal Writing for 
Conferences and 
Meeting 

7 (N=26) 

- How to target a conference 

6 (N— 11) - Familiarizing with proposal templates/criteria 

- To write an proposal for a targeted conference 


The workshop series were designed and structured based on student needs and their calls for facilitating and 
internalizing their academic writing process in their real life context. The following figure (Figure 1) was set up 
at the second year and it represents the core contents and structure of the workshop series. Though research or 
academic writing is not considered a linear process, novice researchers or teachers can easily follow these 
sequences of workshops without losing their research foci. 



Figure 1. Flow chart of the workshop series 


2.4 Research Questions 

With two years’ accumulative quantitative data on all Research Academy’s professional development workshops, 
the investigators asked the following Research Questions (RQs): 

1) Are there any significant mean differences in participants’ overall confidence level before and after each 
workshop and over the two years? 

2) Are there any significant mean differences in participants’ perceptions before and after each workshop and 
over the two years? 

3) To what extent do the participant’s overall confidence and other relevant variables impact their achievement 
test scores? 

3. Method 

3.1 Participants 

The total number of the participants was 492 consisting of COE graduate students who are currently enrolled in 
the southern state university and teachers or practitioners who already graduated from the university. These 
graduate students are all pre- and in-service teachers. The majority of the participants were females (female: 67%, 
male: 33%) and a substantial number of the participants were repeat attendees of the workshops. Their average 
age and years of teaching experience were 36.6 and 7.57, respectively. The participants’ language background 
reflects on the strong Hispanic influence, showing approximately 47% with Spanish as their LI while they also 
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indicated either Spanish or bilingual preferences in reading and writing (16%) and in speaking (32%). Table 2 
shows the demographics of participants participated in the study. 

3.2 Instruments and Data Collection Procedures 

Data collected from each workshop included demographic information, pre-and-post perception survey, 
pre-and-post content assessment designed by the council committee members, and satisfaction survey (See Lu, 
Ward, Overton, & Shin, 2014, for detailed description of the instruments and discussion regarding the 
effectiveness of the workshops by examining pre-and-post content assessment and satisfaction surveys of the 
workshops). Data collection procedures were all similar and instruments were given in order throughout the 
workshops: demographic data, pre-perception survey, pre-content assessment before the workshop session starts, 
and post-perception survey, post-content assessment and satisfaction survey after completing all the workshop 
sessions including PPT presentation, hands-on activities, and students-committee members’ interaction sessions. 
The perception survey consists of a series of questions about specific skills and knowledge which were 
essentially covered or required to gain during a workshop. It also consists of an overall confidence rating in 
which the participants indicated the extent of agreement on a Likert-scale. 

3.3 Data Analysis 

In addition to exploratory data analysis for descriptive statistics, RQ1 and RQ2 were tested through a series of 
paired t-tests for within-subject effects and one-way ANOVA to compare pre- and post-tests and between year 1 
and year 2 data. Regarding RQ3, an exploratory multiple regression analysis was utilized to identity which 
elements mainly influenced the performance of the participants (i.e., age, teaching experience, years in a 
program, pre-and-post overall confidence, and confidence increase, etc.). 


Table 2. Summary of demographic data 


Total 

Workshop Year valid 
# 


Gender Graduate program LI Reading/Writing Speaking 

Teach.Grad. 

years years Ed g 

F M G&C C&I Other EnglishSpanish Other EnglishSpanish Both EnglishSpanish Both 

tech Ed 


Workshop 

exp. 


Yes No 


Journal 

i st 

30 

- 

3.1 

1 

22 

4 

2 

1 

0 

3 

9 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

reading 

2 nd 

63 

39 

9.9 

1.3 

44 

12 

10 

4 

4 

7 

13 

35 

21 

3 

49 

5 

5 

44 

7 

8 

11 

48 

APA 

l St 

64 

32.36 

3.07 

1.3 

33 

9 

12 

1 

7 

13 

10 

23 

17 

2 

32 

3 

7 

27 

6 

9 

18 

22 

writing 

2 nd 

49 

33 

8.1 

1.14 

33 

14 

10 

1 

3 

0 

13 

37 

10 

0 

41 

3 

4 

38 

3 

7 

34 

13 

Research 

1 st 

37 

- 

- 

- 

- 

- 

- 

- 

-- 

-- 

- 

- 

-- 

- 

- 

-- 

- 

- 

- 

-- 

- 

- 

methods 

2 nd 

38 

39 

9.4 

1.5 

26 

12 

4 

2 

5 

0 

11 

28 

14 

0 

33 

1 

4 

28 

4 

7 

35 

3 

statistics 

1 st 

19 

35.88 

7.4 

1.64 

14 

5 

4 

4 

2 

5 

4 

14 

5 


18 

1 


17 

2 


14 

5 


1 st 

43 

.. 

.. 

„ 

„ 

.. 


„ 

__ 

_ 

.. 

.. 

_ 

.. 


_ 

__ 

.. 

.. 

_ 

„ 


Qualitative 

2 nd 

27 

36 

13 

1.71 

22 

5 

8 

0 

2 

3 

15 

15 

10 

1 

22 

2 

1 

20 

4 

1 

19 

7 


1 st 

43 

37.84 

8.76 

1.53 

34 

9 

8 

2 

1 

3 

8 

21 

10 

2 

28 

3 

3 

26 

4 

4 



IRB 

2 nd 

39 

36 

9.32 

1.55 

24 

9 

1 

0 

1 

3 

8 

30 

4 

0 

33 

0 

1 

29 

1 

4 

9 

10 

Proposal 

1 st 

25 

35.59 

6.3 

2.1 

18 

5 

5 

1 


4 

7 

15 

8 

1 

20 

1 

2 

14 

4 

5 

21 

2 

writing 

2 nd 

15 

41 

5 

1.32 

13 

2 

2 

1 

2 

5 

4 

11 

4 

0 

14 

1 

0 

14 

1 

0 

14 

1 


Note. Graduate programs: G&C (Guidance and Counseling), C&I (Curriculum and Instruction), and Others 
(Ed.D., Ph.D., bilingual education, and early childhood education) 


4. Results 

4.1 Overall Confidence 

Participants were asked to rate their own confidence level before and after the workshops. Table 3 shows their 
ratings and mean comparison results. Overall there was significant increase in participants’ level of confidence 
in Journal Reading, APA writing (first year). Research Methods (marginal sig.), Qualitative Research, Human 
Subject Research/IRB, and Preparing Proposals. Statistics Review workshop did not yield confidence increase. 
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Year comparisons indicated participants increased their confidence level in Preparing Proposals and decreased 
their confidence level in APA writing. 


Table 3. The summary of overall confidence results 




No. of 






Year 

workshop 

Year 

Pre mean Post mean 

Paired t-value 


Cohen’s d 

comparison 

valid data 

set 

(SD) 

(SD) 

(df) 

P 

(effect size r) 

(post-test) 









F p 


1 st 

30 

2.47 

3.03 

-2.379* 

.024 

.612 


Journal reading 

(1.01) 

(.81) 

(29) 

(.292) 

— 

2 nd 

— 

— 

— 

— 

— 

— 



1 st 

^>nd 

41 

60 

1.93 

2.44 

-2.548* 

.015 

.256 

.515 


APA writing 

(1.00) 

1.67 

(.98) 

1.45 

(40) 

1.148 

(.250) 

.204 

19.501*** .000 



(.95) 

(1.19) 

(59) 

(.101) 


Research 

1 st 

— 

— 

— 

— 

— 

— 


methods 

2 nd 

16 

2.24 

2.65 

-2.076 

.055 

.600 

— 



(.77) 

(.59) 

(15) 

(.286) 


Statistics 

1 st 

19 

1.63 

1.89 

-.815 

.426 

.245 


1 st 

^>nd 

(.96) 

(1.15) 

(18) 

(.122) 


Qualitative 

12 

2.00 

4.17 

-8.990*** 

.000 

2.972 

__ 



(.74) 

(.72) 

(11) 

(.830) 



1 st 

30 

1.84 

2.88 

-4 011*** 

.000 

1.026 


IRB 

(1.10) 

(.92) 

(29) 

(.456) 

1.279 .262 

ond 

25 

1.40 

2.66 

-7.960*** 

.000 

1.760 



(.68) 

(.75) 

(24) 

(.661) 



1 st 

/->nd 

12 

12 

1.94 

3.13 

-4.062** 

.002 

.000 

2.140 


Proposal writing 

(.66) 

2.00 

(.62) 

4.17 

(11) 

-8.990*** 

(-731) 

2.972 

16.945*** .000 



(.74) 

(.72) 

(ID 

(.823) 



*p<. 05; **/><.01; ***/K.001 


4.2 Results of Perception Survey 
4.2.1 APA Workshop 

The investigators examined each items on the survey used for participants’ perceptions of APA knowledge and 
skills in the APA workshop. Table 4 and Figure 2 show data analysis results. It is shown that there were 
significant increases from pre- to post- surveys. 


Table 4. APA workshop: results of perception survey 


Item Description 

Year 

Pre 

Mean 

(SD) 

Post 

Mean 

(SD) 

Increase 

(%) 

t-value 

(df) 

P 

Year 

comparison 

(posttest) 

F p 



1.93 



-6.256*** 




1 st 


2.62 (.66) 

57.94 


.000 


Major source of errors 


(.89) 



(41) 



1 







3.47 .065 

in APA 


1.67 



-6.874*** 




2 nd 


2.35 (.76) 

59.60 


.000 




(.82) 



(59) 
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1 st 

1.90 

2.50 (.86) 

43.45 

-5.030*** 

.000 

2 Proofreading skills 


(.91) 


(41) 



2 nd 

1.65 

2.15 (.80) 

40.96 

-5.343*** 

.000 




(.82) 


(59) 



1 st 

2.07 



-5.876*** 


Ability to write APA 

3 style 

(.81) 

2.64 (.66) 

42.46 

(41) 

.000 



2 nd 

1.82 

2.25 (.75) 

32.47 

-4.506*** 

.000 




(.89) 


(59) 



1 st 

2.10 

2.67 (.75) 

40.48 

-5.876*** 

.000 

4 Ability to write APA 


(.82) 


(41) 


reference list 

2 nd 

1.88 

2.35 (.84) 

34.46 

-5.350*** 

.000 



(.87) 


(59) 



1 st 

2.55 

2.98 (.72) 

32.54 

-3.344** 

.002 

Ability to find APA 


(.92) 


(41) 


resources 

2 nd 

2.10 

2.62 (.85) 

44.21 

-4.305*** 

.000 



(1.00) 


(59) 



1 st 

1.93 

(1.00) 

2.44 (.98) 

18.52 

-2.548* 

(40) 

.015 

6 Overall confidence 






2 nd 

1.67 

1.45 

-9.43 

1.148 

.256 



(.95) 

(1.19) 


(59) 



4.44* .038 


7.49** .007 


3.812 .054 


5.06* .027 


19.50*** .000 


* p<. 05; **/K.01; ***/K.001 



2nd Year Confidence Differences 




Major source 
of errors 

Proofreading 

skills 

Writing APA 
style 

Writing APA 
reference list 

Fnding APA 

resources 

Overall 

confidence 



■ pre-survey 

O post-survey 



of errors 

skills 

style 

reference list 

resources 

confidence 



■ pre-survey 

\ post-survey 




Figure 2. APA workshop confidence differences 
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4.2.2 IRB Workshop 

The investigators examined each items used for participants’ perceptions regarding Human Subject 
Research/IRB knowledge and skills in the IRB workshop. Table 5 and Figure 3 show data analysis results. It is 
clearly shown that there were significant increases from pre- to post-surveys. 


Table 5. IRB workshop: results of perception survey 


Year 


Item Description 


1 Major concepts of IRB 


Good and bad IRB 

2 

protocols 


Protecting Human 
Subjects 


4 IRB procedures 


5 Resources for IRB 


6 Overall confidence 



Pre 

Post 

Increase 

t-value 


Year 

Mean (SD) 

Mean (SD) 

(%) 

(df) 

P 

1 st 

1.74 

2.81 

94.27 

-7 257*** 

.000 


(.89) 

(.59) 


(33) 


2 nd 

1.65 

2.76 

73.04 

-9.280*** 

.000 


(.69) 

(.74) 


(29) 


1 st 

1.89 

2.76 

72.38 

-4.881*** 

.000 


(.87) 

(.62) 


(33) 


2 nd 

1.74 

2.76 

71.57 

-6.528*** 

.000 


(.79) 

(.74) 


(29) 


1 st 

2.14 

2.93 

81.19 

-4 45i*** 

.000 


(1.06) 

(.68) 


(33) 


2 nd 

2.12 

2.97 

49.26 

-5.221*** 

.000 


(.98) 

(.63) 


(29) 


1 st 

1.91 

2.83 

91.67 

-6.569*** 

.000 


(1.01) 

(.62) 


(33) 


2 nd 

1.62 

2.76 

72.55 

-7.663*** 

.000 


(.70) 

(.78) 


(29) 


1 st 

2.00 

3.00 

93.76 

-6 971*** 

.000 


(1.06) 

(.66) 


(33) 


2 nd 

1.74 

2.82 

62.25 

-8.500*** 

.000 


(.67) 

(.72) 


(29) 


1 st 

1.84 

2.88 

109.44 

-5.308*** 

.000 


(1.10) 

(.916) 


(29) 


2 nd 

1.40 

2.66 

75.00 

-7.960*** 

.000 


(.68) 

(.75) 


(24) 



comparison 

(posttest) 

F p 


.086 .771 


.000 .986 


.077 .782 


.182 .671 


1.239 .269 


1.279 .262 


*/?<.05; **/K.01; ***/K.001 
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2nd Year Confidence Differences 
4 

B 

2 

1 

0 

Major concepts Good and bad Protecting IRB procedures Resources for Overall 
ofIRB IRB protocols Human Subjects IRB confidence 

pre-survey post-survey 
11 pre-survey v' post-survey 



Figure 3. IRB workshop confidence differences 


4.2.3 Preparing Proposals for Conferences and Meeting 

The investigators examined each item used for participants’ perceptions of their proposal preparation ability and 
skills in the Proposal workshop. Table 6 and Figure 4 show data analysis results. It is clearly shown that there 
were significant increases from pre- to post- surveys. 


Table 6 . Preparing proposals for conferences and meeting: results of perception survey 


Item Description Year 


1 st 

j Components of 
proposal writing c 


Differentiating between 1 st 

2 good proposals and bad 

ones 2 nd 

Components of 1 st 

3 field-related proposal 

writing 2 nd 

1 st 

4 Writing a proposal 

writing 2 „d 

Seeking conference 1 st 

5 information and 

resources 2 nd 


Pre 

Mean (SD) 


Post 

Mean 

(SD) 


Increase 
(o/ r 


t-value 

(df) 


1.85 

3.12 

86.66 

-7.042*** 

(. 88 ) 

(.59) 

(25) 

2.00 

3.58 

112.50 

-6 917*** 

(.95) 

(.67) 

(ii) 

1.92 

3.00 

67.33 

-6 499 *** 

(.85) 

(.69) 

(25) 

2.17 

3.67 

101.39 

-5.745*** 

(.94) 

(.49) 

( 11 ) 

1.77 

3.04 

84.00 

-8.935*** 

(.77) 

(. 66 ) 

(25) 

2.17 

3.75 

104.17 

-6 917*** 

(.94) 

(.45) 

(ii) 

1.85 

2.88 

72.67 

-6.429*** 

(.93) 

(.71) 

(25) 

2.00 

3.42 

102.78 

-6.189*** 

(.95) 

(.52) 

( 11 ) 

2.23 

3.19 

58.00 

-6.338*** 

(1.03) 

(.57) 

(25) 

2.92 

3.83 

54.86 

-4.005** 

(1.08) 

(.58) 

( 11 ) 


Year 

comparison 
P (posttest) 

F p 

.000 

4.770* .036 

.000 

.000 

8.957** .005 

.000 

.000 

11.327** .002 

.000 

.000 

5.374* .026 

.000 

.000 

10.376** .003 

.002 
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Item Description 

v Pre „ P / S ' Increase 

Year Mean (SD) (%) 

t-value 

(df) 

Year 

comparison 

P (posttest) 

F p 


j Components of 
proposal writing 

Differentiating between 
2 good proposals and bad 
ones 


Components of 

3 field-related proposal 
writing 

4 Writing a proposal 
writing 

Seeking conference 

5 information and 
resources 


6 Overall confidence 


1 st 
2 nd 
1 st 
2' 
1 st 
2' 
1 st 
2 ' 
1 st 
2' 

1 st 

'-ynd 


nd 


nd 


nd 


nd 


1.85 

3.12 

86.66 

-7.042*** 

(. 88 ) 

(.59) 

(25) 

2.00 

3.58 

112.50 

-6 917*** 

(.95) 

(.67) 

(ii) 

1.92 

3.00 

67.33 

-6 499 *** 

(.85) 

(.69) 

(25) 

2.17 

3.67 

101.39 

-5.745*** 

(.94) 

(.49) 

( 11 ) 

1.77 

3.04 

84.00 

-8.935*** 

(.77) 

(. 66 ) 

(25) 

2.17 

3.75 

104.17 

-6 917*** 

(.94) 

(.45) 

(ii) 

1.85 

2.88 

72.67 

-6.429*** 

(.93) 

(.71) 

(25) 

2.00 

3.42 

102.78 

-6.189*** 

(.95) 

(.52) 

( 11 ) 

2.23 

3.19 

58.00 

-6.338*** 

(1.03) 

(.57) 

(25) 

2.92 

3.83 

54.86 

-4.005** 

(1.08) 

(.58) 

( 11 ) 

1.94 

3.13 

18.63 

-4.062** 

(. 66 ) 

(.62) 

( 11 ) 

2.00 

4.17 

136.11 

-8.990*** 

(.74) 

(.72) 

( 11 ) 


.000 

.000 

.000 

.000 

.000 

.000 

.000 

.000 

.000 

.002 

.002 

.000 


4.770* .036 


.957** .005 


11.327** .002 


5.374* .026 


10.376** .003 


16.945*** .000 


f p<.05\ **/><.01; ***/K.001 


2nd year Confidence Differences 



Components of Differentiating Components of Writing a Seeking Overall 

PW* between good field-related PW proposal writing information and confidence 

and bad resources 


■ pre-survey \ post-survey 

■ pre-survey H post-survey 


Figure 4. Proposal writing workshop confidence differences 
Note. *PW: proposal writing 
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4.3 Results of Achievement Scores (Pre- and -Posttests) 

Table 7 shows the pre- and post-test results for participant achievement assessment in each workshop. From 
pretests to posttests, there were significant gains in their knowledge and skills in all workshops. Effect sizes were 
all medium to large. Year comparison shows participants performed better in year 2 in Research Methods and 
Qualitative Research. 


Table 7. Summary of pre-and post-test results 




No. of 



Paired t-value 

(df) 


Cohen’s d 

Year comparison 


Year 

valid 
data set 

Pre Mean 
(SD) 

Post Mean 
(SD) 

P 

(effect size 
r) 

(Posttest) 

F p 



30 

30.37 

40.79 (10.75) 

-5 921*** 

.000 

.918 


Journal 

1 st 

(11.92) 

(27) 

(.417) 


Writing 

2 n d 

45 

4.60 

10.33 

-15.771*** 

.000 

2.793 




(1.42) 

(2.53) 

(44) 

(.813) 



1 st 

30 

4.78 

11.13 

-5.080*** 

.000 

1.670 


APA 

(4.10) 

(3.48) 

(20) 

(.641) 


<-*nd 

57 

2.34 

3.49 

-3.652*** 

.000 

.690 



Z 

(1.59) 

(1.74) 

(56) 

(.326) 



1 st 

29 

6.45 

7.76 

-3.350** 

.002 

.722 


Research 

(1.94) 

(1.68) 

(28) 

(.340) 

13.844*** .000 

Methods 

^>nd 

35 

6.63 

9.91 

-7.269*** 

.000 

1.265 


Z 

(2.46) 

(2.72) 

(34) 

(.534) 


Statistics 

1 st 

20 

8.75 

13.95 

-5.621*** 

.000 

1.846 


Review 

(2.20) 

(3.32) 

(19) 

(.678) 



1 st 

43 

5.58 

7.40 

-5.802*** 

.000 

.963 


Qualitative 

(2.15) 

(1.59) 

(42) 

(.434) 

8.999** .004 

Research 

^*nd 

12 

6.25 

9.00 

-3.942** 

.002 

1.358 


Z 

(2.22) 

(1.81) 

(11) 

(.562) 



1 st 

43 

8.51 

10.91 

-5.636*** 

.000 

.953 


IRB 

(2.98) 

(1.95) 

(42) 

(.430) 

1.171 .283 

<-*nd 

38 

9.13 

11.38 

-5.567*** 

.000 

1.092 


Z 

(2.07) 

(2.05) 

(38) 

(.479) 



1 st 

26 

6.08 

8.42 

-5.759*** 

.000 

1.067 


Proposal 

(2.47) 

(2.12) 

(25) 

(.453) 

.663 .421 

Writing 

2 n d 

12 

6.25 

9.00 

-3.942** 

.002 

1.358 



(2.22) 

(1.81) 

(11) 

(.562) 



*/K.05; **/K.01; ***/><.001 


Note. The results of year comparison for the journal reading and APA workshops are not available due to 
different task types in year 1 and year 2. 


4.4 Results of Multiple Regression Analysis for APA Workshop 

The investigators conducted exploratory multiple regression analysis on the APA workshop, which contained the 
most valid data points, and the initial results were shown in Table 8. As teaching experience and overall 
confidence were significant predictors of achievement scores, the investigators re-ran the analysis for the 
modified model with step-wise method of entry for the significant predictors. Diagnostics analysis of collinearity 
was conducted and all Tolerance values were greater than .93, indicating high independency of the model’s 
predictors. Table 9 and Figure 5 show the modified model and the regression equation. The model yielded an 
R=,547 (R 2 =,299; SE=3.83), and a significant model fit (F(2, 60)=15.771, MS RegreS sion=231.86, /X.001). 
Therefore, about 30 % of the variance in achievement scores could be explained by the model with predictors of 
teaching experience and overall confidence. 
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Table 8 . APA workshop: regression analysis predicting posttest score (N=63) 


Variable 

Unstandardized Coefficients 

B SE 

Standardized Coefficients 

Beta (P) 

t 

p 

Age 

-.014 .037 

-.038 

-.376 

.708 

Years in graduate program 

.640 .490 

.131 

1.307 

.196 

Teaching experience 

-.210 .069 

-.302 

-3.027* 

.003 

Overall confidence (post-survey) 1.536 .385 

.402 

3.993** 

.000 

Note. Constant=4.379, F(4,58)= 

=8.306***,/K.001, R 2 =.316 




Table 9. APA workshop: regression analysis predicting posttest 

score (N=63) modified model 


Variable 

Unstandardized Coefficients 

B SE 

Standardized Coefficients 

Beta (P) 

t 

P 

Constant 

4.482 .951 


4 713 *** 

.000 

Teaching experience 

-.211 .068 

-.304 

-3 112*** 

.003 

Overall confidence (post-survey) 

1.658 .373 

.434 

4 443 *** 

.000 

Note. Constant=4.482, F(2,60)= 

T5.771***,/K.001, R 2 =299 





Estimated achievement scores = 4.482-0.211 (Teaching experience) + 1.658 (Overall confidence) 


Figure 5. Regression equation 


5. Discussion 

5 .1 General Discussion 

Several points are worth mentioning from this two-year accumulative data. (A) All attendees of the workshops 
were graduate students from different graduate programs in the College of Education in this HSI with some years 
of teaching experience. There were more female than male participants, and most of the participants’ first 
language (LI) is either Spanish or English. Many of the attendees had experience participating in previous 
Research Academy workshops (Table 2). (B) Most workshops successfully increased participants’ overall 
confidence level, such as the journal reading, APA writing (1 st year), research methods (marginal sig.), 
qualitative research, IRB, and proposal writing workshops (Table 3). The medium to large effect sizes indicated 
the practical significance of these workshops. It is reasonable to assume that those who had already learned APA 
style/academic writing in their first year of graduate program did not find the APA workshop particularly helpful 
in increasing their confidence level in APA writing. In addition, the first year’s statistics workshop did not yield 
significant confidence gains, which showed many students’ fear of statistics and a general need for increasing 
their confidence in statistics and strengthening statistics knowledge and skills for graduate students in this HSI. 
(C) When examining participants’ perception of each item from the APA workshop, the investigators found that 
there were significant increases in participants’ ability to detect major sources of errors in APA, write in APA 
style, reference list, find resources, and proofreading skills (Table 4 & Figure 2). (D) When examining 
participants’ perception of each item from the IRB workshop, the investigators found that there were significant 
increases in participants’ understanding of major concepts, IRB protocols, human subject protection, IRB 
procedures, and finding resources for IRB (Table 5 & Figure 3). (E) When examining participants’ perception of 
each item from the Proposal Writing workshop, the investigators found that there were significant increases in 
participants’ ability to determine components of proposal writing and components of field-related proposal 
writing, differentiate between good and bad proposals, write a proposal, and seek conference information and 
resources (Table 6 & Figure 4). (F) The significant increase of participants’ actual achievement test scores not 
only demonstrated their gains in research knowledge and skills through these workshops, but also provided 
evidence to explain their increase in confidence and their positive perception of learning (Table 7). Therefore, 
Latina/o graduate students at this HSI seemed to provide consistent and correct perceptions of their learning in 
regard to their actual learning achievements. (G) The modified multiple regression model revealed that teaching 
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experience and overall confidence were significant predictors of achievement scores (Table 9). Those with fewer 
years of teaching experience tended to obtain better achievement scores in APA and those with higher 
confidence level tended to perform better in achievement tests (Table 9 & Figure 5). 

5.2 Implications 

Shifting from content-based, curriculum-based or strategy implementation activities (Cohen & Hill, 2000; 
Kennedy, 1998) to a more research-based approach, which placed emphasis on how to conduct 
classroom-related or field-based research, the paper demonstrated a new perspective that might be needed to 
examine the effectiveness of professional development designed to improve teachers’ conceptual understanding 
of research knowledge and skills. 

As these graduate students had provided positive perceptions of their learning research-related knowledge and 
skills through these workshops, the results of these effective workshops could potentially benefit not only HSIs 
but also other higher education institutions that emphasize teachers’ research ability or educating future teacher 
researchers. Researchers and practitioners may utilize the study’s model for implementing workshops for 
teachers to learn research-related knowledge and skills and to increase their confidence. 

The outcome from the regression analysis could be considered an expected result, given the nature of learner’s 
perception on the workshop. It is assumed that participants’ confidence level from participating in a workshop 
positively influenced and thus predicted their achievement performance (posttest content assessment). Indeed, the 
higher the overall confidence level a participant had, the better the achievement score was. In addition, the less the 
participants’ teaching experience was, the higher their achievement scores were. 

The results implied that the pre-service or novice teachers were more capable of acquiring and adopting new 
knowledge regarding specific contents from the APA workshop. It might also reflect that pre-service or novice 
teachers are more urged to conduct time-efficient and cost-effective research in their field. Under the circumstance, 
the workshop series provided valuable opportunities for them to learn field-related research techniques and skills. 

5.3 Limitations of the Study 

Since some measurements used in the 2 nd years’ workshops were improved or modified and thus different from 
the 1 st years’, there were some discrepancies that did not allow investigators to make meaningful comparisons. 
Therefore, it would be more ideal to design and implement consistent measurements for longitudinal data 
analysis. In addition, though some participants had participated in more than two workshops or in both years, 
some participants were new and thus were totally different from those who participated in the first years’ 
workshops. This limited some possible repeated-measures continuing data analysis for between year 
comparisons though the study mainly focused on the structure, perception, and effectiveness of the workshops. 
Last, though the study had reported many statistically and practically significant results, obtaining more valid 
data would help further analyses, such as a more comprehensive multiple regression analysis. 

5.4 Future Directions for Research 

Researchers may consider testing participants’ long-term memory by assessing their retention after one week or 
one month of the workshop. As achievement scores were all tested at the end of each workshop, memories 
learners obtained might not be firmly stored in their long-term storage. With another round of test performed 
after one week or one month, the researchers will be able to investigate if these workshops also promoted 
long-term memories on research-related knowledge and skills. In addition, researchers may consider conducting 
more analyses in inferential statistics for other workshops other than the APA/academic writing workshop in the 
future when more valid datasets are collected. Unlike Lawless and Pellegrino (2007) in which they focus more 
on professional development in instructional technologies, future research may incorporate instructional 
technologies into teacher’s PD training and investigate possible differences that may occur between these 
different delivery methods (e.g., Lawless & Pellegrino, 2007). Yet, similar to Lawless and Pellegrino’s (2007) 
reporting knowns and unknowns, future studies may investigate what has not been known in teachers’ 
professional development training. Last but not least, to probe these pre-service and in-service teachers’ thoughts 
and opinions, qualitative inquiries, such as focus groups or interviews, may also help uncover what is needed and 
what aspects of the workshops can be improved or enhanced. 
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6. Conclusion 

The study augmented current literature in teacher’s professional development in research by examining the 
two-year accumulative data from Research Academy workshops held for mainly pre-service and in-service 
teachers in a southern state Hispanic-Serving Institution in USA. They study found that participating graduate 
students’ perceptions of learning through this specific type of professional development workshops, namely the 
research workshops, were positive and effective. The study also found consistent and significant gains of 
confidence level of participants in research skills and knowledge in education. Mean comparisons from the 
two-year data were reported and the journal reading, APA writing, qualitative research, IRB/human subjects 
protection, and proposal writing workshops were among the most successful ones in increasing participants’ 
confidence as well as learning outcomes. 
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